COSE-only ledgers by maxtropets · Pull Request #7772 · microsoft/CCF

maxtropets · 2026-03-25T23:36:15Z

❗ PLEASE CHECK THIS OUT BEFORE REVIEWING THANK YOU VERY MUCH

This continues ☂️: Self-transparent, COSE-only ledgers #7401 journey.
Documentation extended to explain how to cook COSE-only ledgers by enabling the necessary configuration.
To support recovery from all sorts of dubious ledgers and configurations, as well as serve historical queries on them, signature parsing is satisfied when only sees some signature
Signature parsing/verification will also verify all signatures present if they match the required write version

👊 On COSE-only enforcing

This PR deliberately avoids any enforcing and the behaviour is config driven.
Permissive ledger parsing allows flip-flops, which are unavoidable for the mixed network anyway.
As we settle on the enforcement strategy, we can have another PR for it.
- For instance, we may have a proposal to set COSE-only flag once the upgrade has come through, and then any attempt to Join/Recover in Dual mode will fail unless operators break glass and unset the flag or smth.
- We can also say we'll leave it up to the operators.

⚠️ On LTS

It is not supported to have mixed 6.x and 7.x.COSE.only
The required path is described in the documentation and needs go through 6.x -> 7.x, 7.x -> 7.x.COSE.only
This greatly simplifies LTS work and I'll insist we keep it this way unless any strong argument against is found.

Copilot

Pull request overview

Adds configuration-driven support for COSE-only ledger signatures across the CCF runtime, endpoints, SDK, and test suite, enabling operation and recovery on ledgers where only COSE signatures are emitted.

Changes:

Introduces ledger_signatures.mode (Dual/COSE) in node config + schema/docs/samples.
Updates history/signature/receipt plumbing to tolerate ledgers containing either signature type and to build receipts accordingly.
Adds /receipt/cose endpoint and extends E2E/recovery tests to exercise COSE-only and upgrade scenarios.

Reviewed changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
tests/recovery.py	Adds recovery scenario validating COSE-only ledgers across snapshot + non-snapshot recovery.
tests/infra/remote.py	Threads `ledger_signature_mode` through config rendering for nodes.
tests/e2e_operations.py	Adds COSE-only upgrade test and receipt verification using COSE receipts.
tests/e2e_logging.py	Updates receipt tests to handle COSE-only mode and introduces COSE receipt helper(s).
tests/config.jinja	Emits `ledger_signatures.mode` into generated node configs.
src/service/internal_tables_access.h	Derives last signed root from serialised Merkle tree rather than node signatures table.
src/node/tx_receipt_impl.h	Makes non-COSE signature and root optional in internal receipt representation.
src/node/test/history.cpp	Extends history tests for Dual vs COSE-only modes and table population expectations.
src/node/signature_cache_subsystem.h	Treats cached signature as complete if either signature type is present (plus tree).
src/node/signature_cache_interface.h	Makes cached signature fields optional to support COSE-only operation.
src/node/node_state.h	Recovery/join paths now extract view/seqno info from COSE signature when traditional signature absent.
src/node/history.h	Adds ledger signature mode to signing identity; emits signatures conditionally (Dual vs COSE-only); adjusts verification behavior.
src/node/historical_queries.h	Builds receipts even when only COSE signature is available (traditional signature optional).
src/node/historical_queries_adapter.cpp	Updates receipt description/serialization to handle optional signature/root; tightens COSE receipt building conditions.
src/node/gov/handlers/acks.h	Computes acked state digest from serialised Merkle tree (compatible with COSE-only).
src/kv/kv_types.h	Extends TxHistory interface to accept `ledger_signature_mode`.
src/kv/deserialise.h	Accepts signature transactions containing either signature type (plus serialised tree).
src/endpoints/endpoint_registry.cpp	Builds receipt from cached signature when traditional signature may be absent.
src/endpoints/common_endpoint_registry.cpp	Adds `/receipt/cose` endpoint returning COSE receipts with `application/cose` content type.
src/common/configuration.h	Adds JSON enum mapping + parsing for `ledger_signatures.mode`.
samples/config/start_config.json	Adds `ledger_signatures.mode` sample default (`Dual`).
samples/config/recover_config.json	Adds `ledger_signatures.mode` sample default (`Dual`).
samples/config/join_config.json	Adds `ledger_signatures.mode` sample default (`Dual`).
python/src/ccf/ledger.py	Counts signature transactions when either signature table is present.
python/src/ccf/cose.py	Allows skipping claim digest verification when `claim_digest` is omitted.
include/ccf/node/startup_config.h	Introduces `CCFConfig::LedgerSignMode` + `ledger_signatures.mode` field.
doc/schemas/node_openapi.json	Documents `/node/receipt/cose` in checked-in OpenAPI schema.
doc/schemas/app_openapi.json	Documents `/app/receipt/cose` in checked-in OpenAPI schema.
doc/operations/configuration.rst	Documents COSE-only mode configuration and upgrade/recovery guidance.
doc/host_config_schema/cchost_config.json	Adds schema for `ledger_signatures.mode`.
CHANGELOG.md	Notes new `ledger_signatures.mode` configuration option.

tests/e2e_logging.py

src/endpoints/common_endpoint_registry.cpp

doc/schemas/node_openapi.json

doc/schemas/app_openapi.json

python/src/ccf/cose.py

doc/operations/configuration.rst

Copilot

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 3 comments.

src/node/historical_queries_adapter.cpp

doc/schemas/app_openapi.json

Copilot

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 6 comments.

tests/e2e_logging.py

src/node/historical_queries.h

python/src/ccf/ledger.py

src/node/signature_cache_interface.h

include/ccf/ds/openapi.h

doc/operations/configuration.rst

Copilot

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 8 comments.

src/node/historical_queries_adapter.cpp

src/node/node_state.h

Copilot · 2026-04-10T13:22:31Z

src/node/node_state.h

+            auto receipt = cose::decode_ccf_receipt(lcs.value(), false);
+            auto tx_id_opt = ccf::TxID::from_str(receipt.phdr.ccf.txid);
+            if (!tx_id_opt.has_value())
+            {
+              throw std::logic_error(fmt::format(
+                "Failed to parse TxID from COSE signature: {}",
+                receipt.phdr.ccf.txid));
+            }
+            sig_view = tx_id_opt->view;


During public ledger recovery, the COSE-signature fallback path calls cose::decode_ccf_receipt() without catching cose::COSEDecodeError. A malformed COSE signature entry would throw and abort recovery via an uncaught exception, rather than producing a controlled recovery failure with a clear message. Consider wrapping this decode+TxID parse in a try/catch and converting errors to std::logic_error with the seqno/txid context.

Suggested change

auto receipt = cose::decode_ccf_receipt(lcs.value(), false);

auto tx_id_opt = ccf::TxID::from_str(receipt.phdr.ccf.txid);

if (!tx_id_opt.has_value())

{

throw std::logic_error(fmt::format(

"Failed to parse TxID from COSE signature: {}",

receipt.phdr.ccf.txid));

}

sig_view = tx_id_opt->view;

try

{

auto receipt = cose::decode_ccf_receipt(lcs.value(), false);

auto tx_id_opt = ccf::TxID::from_str(receipt.phdr.ccf.txid);

if (!tx_id_opt.has_value())

{

throw std::logic_error(fmt::format(

"Failed to parse TxID from COSE signature at seqno {}: {}",

last_recovered_idx,

receipt.phdr.ccf.txid));

}

sig_view = tx_id_opt->view;

}

catch (const cose::COSEDecodeError& e)

{

throw std::logic_error(fmt::format(

"Failed to decode COSE signature at seqno {}: {}",

last_recovered_idx,

e.what()));

}

Todo: make sure

COSE receipt endpoints handles that, and 404 properly too

all decode errors are handled if needed

src/node/historical_queries.h

src/kv/deserialise.h

src/node/signature_cache_subsystem.h

maxtropets self-assigned this Mar 25, 2026

maxtropets mentioned this pull request Mar 25, 2026

☂️: Self-transparent, COSE-only ledgers #7401

Open

13 tasks

maxtropets changed the title ~~[WIP COSE-only ledgers~~ [WIP] COSE-only ledgers Mar 25, 2026

maxtropets force-pushed the f/COSE-only-sig branch 2 times, most recently from 3287ca3 to 1676517 Compare March 30, 2026 13:32

maxtropets force-pushed the f/COSE-only-sig branch 6 times, most recently from c9de792 to 076a647 Compare April 8, 2026 12:55

maxtropets added run-long-test Run Long Test job and removed run-long-test Run Long Test job labels Apr 8, 2026

maxtropets force-pushed the f/COSE-only-sig branch 3 times, most recently from 76e4e79 to 20e9eb6 Compare April 9, 2026 12:47

maxtropets added the run-long-test Run Long Test job label Apr 9, 2026

maxtropets force-pushed the f/COSE-only-sig branch 5 times, most recently from a709b2a to 37db60c Compare April 9, 2026 14:23

maxtropets changed the title ~~[WIP] COSE-only ledgers~~ COSE-only ledgers Apr 9, 2026

COSE-only ledgers

f980687

maxtropets force-pushed the f/COSE-only-sig branch from 37db60c to f980687 Compare April 9, 2026 14:46

maxtropets marked this pull request as ready for review April 9, 2026 14:46

maxtropets requested a review from a team as a code owner April 9, 2026 14:46

Copilot AI review requested due to automatic review settings April 9, 2026 14:46

Copilot started reviewing on behalf of maxtropets April 9, 2026 14:47 View session

Copilot AI reviewed Apr 9, 2026

View reviewed changes

eddyashton reviewed Apr 9, 2026

View reviewed changes

doc/operations/configuration.rst Outdated Show resolved Hide resolved

Review

462c454

maxtropets removed the run-long-test Run Long Test job label Apr 10, 2026

maxtropets requested a review from Copilot April 10, 2026 11:37

Copilot started reviewing on behalf of maxtropets April 10, 2026 11:41 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

src/node/historical_queries_adapter.cpp Show resolved Hide resolved

src/node/historical_queries_adapter.cpp Show resolved Hide resolved

doc/schemas/app_openapi.json Outdated Show resolved Hide resolved

Review

8dbdcba

maxtropets requested a review from Copilot April 10, 2026 12:42

Copilot started reviewing on behalf of maxtropets April 10, 2026 12:45 View session

Tidy, tidy, tidy...

a235b84

Copilot AI reviewed Apr 10, 2026

View reviewed changes

Comment

fac0afe

maxtropets requested a review from Copilot April 10, 2026 13:06

Copilot started reviewing on behalf of maxtropets April 10, 2026 13:09 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

maxtropets added the run-long-test Run Long Test job label Apr 10, 2026

Conversation

maxtropets commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

maxtropets Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

maxtropets commented Mar 25, 2026 •

edited

Loading